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Abstract 

For a skip- free Markov process on Z + with generator matrix Q, we 
evaluate the joint Laplace transform of the occupation times before 
hitting the state n (starting at 0). This Laplace transform has a very 
straightforward and familiar expression. We investigate the properties 
of this Laplace transform, especially the conditions under which the 
occupation times form a Markov chain. 

1 Introduction 

Consider a process {X t } t > on {0, 1, 2, } with generator Q := ((qij))o<i,j< c 

satisfying 

qi : i+i > 0, qij — if j > i + 1 and q. L j = for all i > 0. (1) 

j=o 

Such processes are called skip-free (to the left) processes. Note that birth 
and death processes are special cases of the process described above. Let 
T n be the first time when the process reaches n. As usual, 1 % T denotes the 
amount of time {X t } t > spends in state i before it reaches n. Let Q n denote 
the upper n x n submatrix of Q. We are interested in the joint distribution 
of {/T n } <i< n -i Parting at 0. We prove that 

I Qn | 



E 



\D-Q n \ 



(2) 
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where di > 0, z = 0, 1, ...,n — 1 and D is a diagonal matrix with diagonal 
entries di, i = 0,1, ...,n — 1. This formula generalizes the results of Kent 
[3] about birth and death processes. Using the identity (j2J) we prove that 
{^T n jo< < n -i * s a Markov chain iff Q n is a tridiagonal matrix. For these pur- 
poses, we will derive an identity for the joint Laplace transfrom of occupation 
times for a general class of Markov processes and then specialize to skip-free 
processes. 

2 An identity for a general class of Markov 
processes 

Let {X t } t > be a Markov process with finite state space X. Let Q = 
((qxy))x,yex be the corresponding generator matrix. 

Let Si be the random time corresponding to the i th transition for the 

Markov process {X t } t > , i = 1,2,3, Set So = 0. Let {li}j>o := 

{^sJj>o be the embedded discrete-time Markov chain with one step transi- 
tion probabilities 

Pxy = for x ^ y, Vxx = 0. 

Qxx 

Let P := ((p X y))x,yex and let Q dmg denote the diagonal matrix with diagonal 
entries same as Q. Then, 

Q = Q*<*(I-P). (3) 

As usual, let 

1" ■= [ ^{x 8 =x}ds, t>0,xeX 
Jo 

denote the occupation time of the Markov process {X t }t>o in the state x till 
time t. 

We describe a typical path of {X t }t>o- The process starts at an initial 

state Y . The process stays at Y n during [S n ,S n+ i) for n = 0,1,2, 

and at a random time £ = S^+i jumps from the state Y v to a "cemetery" 
A (not included in X), and stays there forever. (Here £ takes non- negative 
real values and rj takes non-neagtive integer values). The analysis presented 
below requires the assumptions that 
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(i) Q 1 exists. 

(ii) P x (r) < oo) = 1 Vx G X. 

Let d = {d u } ue x be arbitrary with d u > 0. Let D denote the diagonal 
matrix with diagonal entries {d u } u( zx- Note that —(D — Q) is the generator 
of a Markov process {X t } t > with the same structure as {X t }t>o except that 
at every state x G X, there is an additional killing rate of d x . Let {Y m } m > 
be the embedded discrete-time Markov chain corresponding to {X t } t > . Let 
us establish the change of measure formula from {Y m } m > Q to {Y m } m > . 

Px{?l = yi, % = J/2, ■•, % = Vn) 



n 



n 



~%i-\Vi 



I lyi-lVi-l + ^W-l 



(with ?/ = ^) 



Px{^i = Y 2 = 2/2, K = y n } 



Hence, 
E x [F(Y 1 ,Y 2 ,...,Y n )]=E ll 



n 



F(Y!,Y 2 ,...,Y n 



(4) 



for each bounded Borel measurable function F. 

We evaluate E x je"^^ 1 which is the joint Laplace transform of the oc- 
cupation times {1^o}ugx evaluated at d = {d u } ue x- The argument is inspired 
by Dynkin [TJ. 
Note that, 



Hence, 



i=0 



J^E, [ e -£?=o<*n^+i-Si)i {j3=n 

n=0 

00 

n=0 



e -Et„4 i( %-^ Wn+i=A} 
ye* 
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oo 



y&X n=0 



J2J2 E * n 

y&X n=0 L \i=0 



{Y n =y,Y n+ i=A} 



- £™=o ^(Si+i-Si) 



{Y m } 



m>0 



1 



{Y n =y,Yn+i=A} 



4y„= y ,y n+ i=A} 



The previous equality follows from the fact that conditioned on {Y m } m > , 
the intermediate transition times {S^+i — Si}i>o are independent and have 

Exponential(—qYiYi) distribution for i — 0, 1, 2, 

By Markov property and fflfy, 



[e 



n=0 g ™ + ^ 



■E 9 



n 



-qv^Yi-! 



s/eA" n=o q yy + d y 
r r 00 

E ~%v e ^ I 1 



E/l,,, „ } ]p( y ,A) 



1 



P{Y 1 = A\Y = y) 



{Xs=y} 



ds 



J2-Qyy(D-Qr\x,y) P {y,A) 

y ex 



Q\ 



\D-Q\ 



Qyy 

\ytx 



\{D-Q) 

\-Q\ 



-P(2/, A ) 



Here, |(-D — Q) | represents the determinant of the matrix obtained after 
removing the y th row and the X th column from D — Q, multiplied by —1 if 
the x th row and the y th row of D — Q differ by an odd number of rows. Thus, 
the joint Laplace transform of {V^u&x is given by 



E x [e 



\-Q\ 
\D-Q\ 



E 



\{D-Q) 



( x ,y) I 



Q\ 



(5) 



In [2], ([5]) is derived using Kac's moment formula. Note that we did not 
require any reversibility assumption on Q in the above argument. 
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3 Application to skip-free processes 



Let us investigate the case of skip-free processes with generator Q given 
by ([1]). We are interested in the joint distribution of {l l T n } 0<i<n _i starting 
at state 0. It follows by the structure of Q that the joint distribution of 
{^T„}o< < n -i Parting at is the same as the joint distribution of the infinite 
occupation times of a Markov chain with generator matrix Q n , starting at 0. 
Hence, by ()5]), for arbitrary di > 0, i — 0, 1, n — 1, 



E 



Qn | 



\D - Q r , 



'n-1 

E 



[D-Q r 



\(0,j) 



n-1 



Qr 



i=0 



1 - Qn\ 


\(D- 


-QnT^ 


\D - Qn 




Qn 





n-1 



i=Q 



The previous equality follows from the fact that 



n-1 



qji — for j < n — 1. 



i=0 

Note that if we remove the (n — I)*' 4 row and th column of D — Q n , the 
resulting matrix is upper triangular with diagonal entries not depending on 
di, i — 0, 1, n — 1. Hence, 

1 



En 



2^=0 a i L T n 



oc 



\D-Q n \ 

By substituting di = for i = 0, 1, n — 1, we get, 



En 



\D-Q n \ 



Let us look more closely at the joint Laplace transform in (T2J). Note that, if 
Qn 1 '■= ((qn))o<i,j<n-h then 



En 



— for di > 0, < z < n- 1. 



Hence, the marginal distribution of l % Tn is Exponential ) for < i < 



n — 1, which means the joint distribution of {/T n } <i< n -i 
exponential distribution. 



is a multivariate 
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It is well known that if 77,77 MVN n (0,TH), i.e. 77 = (771,772,...,^) 
and 77 = (7)1,772, ...,7~7n) are multivariate normal with mean and covariance 
matrix S, then 



E 



for dj > 0, 7 = 1,2,..., n. 



Note that if M = jDE _1 JD -1 , where D is a diagonal matrix, then 

is- 1 ] |M| 
(D + S- 1 ! ~ fD + Mf" 

If Q is reversible with respect to a probability measure 7r, it is possible to 
symmetrize Q by diagonal conjugation, and hence we can identify the joint 
distribution of {^r„} <j< -1 w ^ n the sum OI " componentwise squares of two 
independent MVN n (0, (— random vectors multiplied by |. Q* is 
given by 

Q* n (i,j)=q ij J^, 0<i,j<n-l. 

But a generator for a skip-free process is diagonally conjugate to a symmetric 
matrix if and only if the process is a birth and death process. We refer to 
skip-free processes which are not birth and death processes as strictly skip- 
free processes. Hence, if Q n is not tridiagonal, then the joint distribution of 
VT n So<i< -1 can no ^ ^ e identified with componentwise sums of squares of 
independent multivariate normal random vectors multiplied by ~. 

4 A complex Gaussian measure 

If A is real non-symmetric and positive definite, we still can interpret 

Ml 

<Md) = - ^ for d= (di, d 2 , d n ) with dk > 0, % = 1, 2, n, 

as the Laplace transform of a signed measure. This will be proved using the 
methods in [3]. 

Let C := j4+ 2 4T and B := A ~^ T . Clearly, C is symmetric, B is skew- 
symmetric and A = C + B. Define the measure \i& on C n by 

g?/m(z) = ( e a - + e a - J dz, 
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where z = x + iy and dz = YYj=i YYj=i ^Vr Note that z T Az and z T Az 
are complex conjugates, hence /ia is a real valued (although possibly signed) 
measure. Also, the transformation y — > — y (where z = x + iy) gives, 



e 2 dz 



2 dz. 



To avoid confusion, we clarify that f c „ f{z)dz stands for J Rn J Rn f(z)dxdy. 
Hence, 



e 2 dz 



(x+iy)- 1 (C + B)(x-iy) 

e 2 axrfy 



n J urn 



~dx.dy 



n ./ran 



x J Cx+x J SO B x x 



x J Cx+x J BC _1 B J x 



27r) n |Cr^ x 



(6) 
dy ) dx 

(7) 



2tt / e 



x T (c+ac~ 1 a T )x 



2n , _ l 

2tt) |c|-5 c + bc -1 ^ 5 



dx 



2/i 



2tt |c|-5|q-3 



I-C-vBC-vC-vBC-? 



2n 



2tt |CMq 



I-C-vBC-* 



I + C~*BC~ 



Note that ([6]) follows from the fact that C is symmetric and I? is skew- 
symmetric, and (EJ) follows by using properties of the multivariate normal 
distribution. Continuing further, 



2 ( \/2tt 
2 
2 



2/i 



2 " , m , i 
T|~2 



27T) |y4 

2/i 



)zn 
14 



-i 



If we define the measure \i* A on C n by 



A*i(o 



2(v^) 2n |A| 



then ^(C n ) = 1. 

Let D denote a diagonal matrix with non- negative entries d\, d,2, d n . Then, 



./:• J : 



In 

C 



VD + A(C n ) 



2(V2^) 2n \A\-i 
2 (V^) 2n \D + A]- 1 
2(V2P) 2n \A\-t 



Hence, 



e -i^^iVi(z) = |J^L 



Let fi A s denote the measure induced on 0^-, ^y-j by fi* A . Since fj, A s 

is a signed measure, we can break it up as n A bs = /J, A S,+ — /j, a s ~ , where n A s,+ 
and n A s '~ are non-negative measures. One wonders for which A, [J- A s '~ = 0, 
so that n A s is a probability measure. If A is the upper nxn submatrix of the 
negative of the generator of a strictly skip-free Markov process, then it follows 
by ([2]) and the uniqueness of Laplace transform for a bounded measure, that 
fi A s is a probability measure. 



5 The Markov property 

We now undertake the task of ascertaining the conditions under which |„ . , 

° L J-n J 0<i<n— 1 

satisifes the Markov property for n > 3. 

Lemma 1 J/Q^ (u^/i Q as in §T\)) is tridiagonal, then {^r n } 0< < _ x satisfies 
the Markov property. 
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Proof Since Q n is tridiagonal, we can obtain a symmetric matrix Q* n from Q n 
by diagonal conjugation. It follows that 



0<i<n-l 2 ' 



where, rj, 77 ~ MVN n (0, (— Q*) ) and 77, 77 are independent. By the standard 
properties of Gaussian Markov random fields developed in [5], we get that 



Vk I (rjk-i,Vk-2, Vi) = Vk I Vk-i V2 < k < n. 
Let t > be fixed. Then, 



(8) 



E 
E 
E 



E 



r7 fe _i,r/ fc _ 2 , 171, 77^, ...,r/i 



2 1 ~2 2 1 ~2 



2 1 ~2 2,-2 



r ,fc I r) k -l,T]k-2, 

The previous equality follows by the independence of 77 and rj. By OH]) we get, 
-t(vl+v 2 k ) 



E 
E 
E 
E 



2 1 — 2 2,-2 



%-l 



%-l 



-trj? I 2 1 ~2 2 1 ~2 



e *^ I ?7fc-i 



E 
E 



e ^ I 77 fe _i,77 fe _2, ...,fj 1 ,rj k -i, ...,771 



2 , ~2 2,-2 
%-l +%-!,-, % +77l 



2 1 — 2 2 1 ~2 

+ %_!,..., 77l +77l 



< "■ I Vk-1 

The previous equality follows by the fact that r\ and fj are i.i.d. Note that, 

Vk I »7fc-i ~ N(c 1 r] k ^ 1 ,c 2 ) and % | r/ fc „i ~ N(c 1 fj k -i,c 2 ) 
for some constants c\ and c 2 . This gives, 



2 t 2 



2 t -2 
c l"Jfc-l 



E 



Vk-i 



g 2c 2 t+l 

VScst + 1 



and E 



Vk-l 



g 2c 2 t+l 



(9) 



Combining everything, 



E 



E 



2c 2 t+l 



2c 2 t + 1 



r,2 



2c 2 t+l 



2c 2 t + 1 



Thus, the conditional Laplace transform given r^_ x + r{l-ii •••; ? 7i + Vi * s a 
function of + fjl_ v Since, 



{ Z Uo<i<n-l= 2^ + ^' 

it follows that 

I f /rp . Jjrp ..... Inn 1 \rp I /rp VI A? ?X 1. 

n 1 V -£ n ; J-n 1 ' - L n / J-n ' In — — 

Hence proved. 

We now prove the converse. 

Lemma 2 IfQ n (with Q as in $Jty) is not tridiagonal, {^T n }o<j< n -i ^ oes no ^ 
satisfy the Markov property. 

Proof We first look at the scenario where (Xi, X 2 , X 3 ) are non-negative ran- 
dom variables with joint Laplace transorm given by 



j£ y e -{d 1 X 1 +d,2X 2 +d s X 3 )^ 

where, 



-A\ 
\D-A\ 



■ 



-a u aw 
A | <x 21 -a 22 a 23 

a 31 O32 — a 33 

with ai 2 , a 2 3, 031, a 22 , 033 > 0, a 2 i,a3 2 > and row sums less than zero or 
equal to zero. It follows that, 





\-A\ 




di + a 22 — fi 2 3 




— ^3 2 d 3 + 033 



where a 22 = a 22 - ™, fi 23 = a 32 = ^a 23 a 32 + a«i . It follows that, 

(x 2 ,x 3 )± 1 -( v l v l) + 1 -(f ] l,f 1 i) } 

where 77,77 ~ MVN 2 (0, (— A*) -1 ) are independent with 

j^* _ I — S 22 a 23 

S32 — a 33 
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By 



Note that, 



E [e 



-d 3 X 3 



x 2 ] 



a-33 



d 3 + a 33 



( a 23) d 3 x 2 
e a 33C d 3+°33) . 



X 3 | X 2 , X! — x 3 \ x 2 

& E [E [e~ d:iX3 I X 2 ,X 1 ] e -( dlXl+d2X2 )] = E [E [e - *^ | X 2 ] e -( d ^+ d ^ x ^ yd u d 2 ,d 3 > 



A\ 



\D-A\ 
I - A\ 



E 



Q 33 c -( dlXl +d*X 2 ) 



d 3 + a 33 
a 33 



with c?2 — d 2 + 



id 23 ) 2 d 3 



033(4 + 033! 



Vdi, d 2 , d 3 > 



|D - A| d 3 + a 33 (di + an)^ + ^22)033 - (di + a n )a 23 a 32 - a 12 a 2i a 33 - a X2 a 23 a 3l 
Wt,d 2 ,d 3 > 

After expanding the determinant and cancelling terms from both sides we 
obtain, 



x 3 \ X 2 , Xi — x 3 1 x 2 

(di + a u )a 12 a 23 a 31 d 3 (d 3 + a 33 )a 12 a 23 a 3 i 
<^> 

033 a ll a 33 

^> ai 2 a 23 a 3 i(did 3 - a n a 33 ) = Vdi, d 2 , d 3 > 0, 



W u d 2 ,d 3 > 



which is not true, because ai 2 ,a 23 ,a 31 > by our assumptions. Hence, 
(Xi, X 2 , X 3 ) is not Markov. 

We now return to The scenario considered above immedi- 

ately gives the required result Tor n — 3. But it also helps us in the general 
case. Since Q n is not tridiagonal, we can find the smallest i such that > 
for some j < i—2. Let us call it as Iq and evaluate the joint Laplace transform 
of (#- 2 ,#-\#J. By®, 



E 



|Z>' 3 - Qr 



^0 



l) th and Zg ft entries 



where D 10,3 is a diagonal matrix with the (i — 2 
given by d\,d 2 and <i 3 respectively and other diagonal entries being 0. We 
now reduce the determinant |D* ' 3 — Q n \ to a managable form by performing 
appropriate row and column operations. Subtract a multiple of the first 
column from the second column to make the (1, 2) th entry zero. Successively 
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subtract a multiple of the (j — l) th column from the j th column to make the 
(j — l,j) th entry 0, for j < i — 2. The upper (z — 3) x (i — 3) submatrix 
is now upper triangular. Similarily, subtract a multiple of the n th row from 
the (n — l) th row to make the (n — l,n) th entry 0. Successively subtract a 
multiple of the j th row from the (j — l) th row to make the (j — l,j) th entry 0, 
for j > i + 1. The lower (n — i ) x (n — i Q ) matrix is now upper triangular. 
The fact that Q is a generator matrix as in ([T]) ensures that all the diagonal 
entries are positive. Hence, 



\D 



Qn\ 



di — qi ~2,i -2 —qi -2,i -i 

— Qi -l,i -2 &2 — Qi -l,i -l 





— 9«o-li«o 
^3 Qio,io 



with c > 0. 



Again, Q being a generator matrix as in ([T]) along with q io j > for some 
j < i — 2 ensures that 



i,3 



qi -2,i -2 qi -2,i -i 

qi -l,i -2 qi -l,io-l Qi -l,io 
Qi ,i Q ~2 Qi ,i -1 <?io>«o 



is a generator matrix with q io ^ 2 ,i -i; 1i -i,io, 1i ,io-2 > and row sums less 
than zero or equal to zero. Hence, 



E 



+d 2 tS +d 3 f° 

J-n J-n J-n 



Q 



«o,3| 



\D 3 -Q 



io,3 I 
n 



where D 3 is a diagonal matrix of dimension 3 with diagonal entries di,d2,ds. 
The Laplace transform of {pf.~ , , l l £ n ) satisfies the conditions of the sce- 
nario considered at the beginning of this proof. Hence, {^t~ 2 ^t^ 1 ^T n ) ^ s 
not Markov, which is enough to conclude that {l 1 ^ \ n . is not Markov. 

° J-n ) 0<i<n— 1 

This completes the proof. 
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